Touch event model

ABSTRACT

A device with a multi-touch panel and a computer readable medium storing software for displaying a user interface on the multi-touch panel detects a touch at the multi-touch panel. The device sends to the software a touch data structure for defining a state of the touch at a particular time. The touch data structure includes: a location field indicating a location where the touch was or is being received on the multi-touch panel; a phase field defining a phase of the touch at the particular time, the phase of the touch being selected from a plurality of predefined phase state values; and a view field indicating a view, of a plurality of views corresponding to portions of the displayed user interface, that corresponds to the location where the touch was or is being received on the multi-touch panel.

RELATED APPLICATIONS

This application is a continuation application of U.S. patentapplication Ser. No. 15/432,746, filed Feb. 14, 2017, which is acontinuation application of U.S. patent application Ser. No. 15/197,704,filed Jun. 29, 2016, now U.S. Pat. No. 9,690,481, which is acontinuation application of U.S. patent application Ser. No. 14/171,680,filed Feb. 3, 2014, now U.S. Pat. No. 9,389,712, which is a continuationapplication of U.S. patent application Ser. No. 12/042,318, filed Mar.4, 2008, now U.S. Pat. No. 8,645,827. All of these applications areincorporated by reference herein in their entireties.

FIELD OF THE INVENTION

This relates to multi-point and multi-touch enabled devices in general,and more specifically to recognizing single and multiple point and touchevents in multi-point and multi-touch enabled devices.

BACKGROUND OF THE INVENTION

Multi-touch enabled devices are known in the art. A multi-touch enableddevice is a device that can sense multiple touches at the same time.Thus, a multi-touch enabled device can, for example, sense two touchevents that take place simultaneously at two different positions on amulti-touch panel and are caused by two fingers being pressed down onthe panel. Examples of multi-touch enabled devices are discussed in U.S.patent application Ser. No. 11/649,998, entitled “Proximity andMulti-Touch Sensor Detection and Demodulation,” filed on Jan. 3, 2007and hereby incorporated by reference in its entirety. Multi-pointenabled devices define a more general set of devices that includemulti-touch enabled devices as well as similar devices such as themulti-proximity sensor devices discussed in U.S. patent application Ser.No. 11/649,998 mentioned above.

While the benefits of multi-touch enabled interfaces are known, thesedevices can present some interface design challenges. Existing interfacedesign conventions have assumed a single pointing user input device thatspecifies a single location at a time. Examples include a mouse or atouch pad.

More specifically, many existing graphical user interface (GUI) systemsprovide user interfaces in which various portions of a display areassociated with separate software elements. Thus, for example, portionsof a display can be associated with a window, and the window can beassociated with a specific software application and/or process. A mousecan be used to interact with the window and the application or processassociated with that window. The mouse cursor can then be moved toanother window to interact with another application or process. Becauseonly a single pointing device is used, interaction with only a singlewindow and application or process can occur at a time.

The assumption of a single interaction with a window at any one time cangreatly simplify user interface design. The application and/or processrunning within a window can operate under the assumption that a detectedinteraction with that particular window is the only input beingreceived. Thus, the application and/or process need not concern itselfwith the possibility of other user interactions occurring in otherportions of the display outside that window. Furthermore, a window canbe additionally partitioned into various elements, wherein each elementis associated with a specific portion of the window. Each element can beimplemented by a separate software element (e.g., a software object).Again, each software object can process interactions that occur in itsassociated area without concerning itself with interactions that may besimultaneously occurring elsewhere.

On the other hand, if a multi-touch interface is being used, two or moretouch events can simultaneously occur at different portions of thedisplay. This can make it difficult to split the display into differentportions and have different independent software elements processinteractions associated with each portion. Furthermore, even if thedisplay is split up into different portions, multiple touch events canoccur in a single portion. Therefore, a single application, process orother software element may need to process multiple simultaneous touchevents. However, if each application, process or other software elementneeds to consider multiple touch interactions, then the overall cost andcomplexity of software running at the multi-touch enabled device may beundesirably high. More specifically, each application may need toprocess large amounts of incoming touch data. This can require highcomplexity in applications of seemingly simple functionality, and canmake programming for a multi-touch enabled device generally difficultand expensive. Also, existing software that assumes a single pointingdevice can be very difficult to convert or port to a version that canoperate on a multi-point or a multi-touch enabled device.

SUMMARY OF THE INVENTION

Embodiments of the present invention are directed to methods, software,devices and APIs for defining touch events for application levelsoftware. Furthermore, some embodiments are directed to simplifying therecognition of single and multiple touch events for applications runningin multi-touch enabled devices. To simplify the recognition of singleand multiple touch events, each view within a particular window can beconfigured as either a multi-touch view or a single touch view.Furthermore, each view can be configured as either an exclusive or anon-exclusive view. Depending on the configuration of a view, touchevents in that and other views can be either ignored or recognized.Ignored touches need not be sent to the application. Selectivelyignoring touches can allow for simpler applications or software elementsthat do not take advantage of advanced multi touch features to beexecuted at the same device (and even at the same time) as more complexapplications or software elements.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram of an input/output processing stack of an exemplarymulti-touch capable device according to one embodiment of thisinvention.

FIG. 2A is a diagram of an exemplary multi-touch enabled deviceaccording to one embodiment of this invention.

FIG. 2B is a diagram of another exemplary multi-touch enabled deviceaccording to one embodiment of this invention.

FIG. 3 is a diagram of an exemplary multi-touch display according to oneembodiment of this invention.

FIG. 4 is a flow chart showing an exemplary method of operation of themulti-touch flag according to one embodiment of this invention.

FIGS. 5A and 5B are flowcharts showing an exemplary method of operationof the exclusive touch flag according to one embodiment of thisinvention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

In the following description of preferred embodiments, reference is madeto the accompanying drawings which form a part hereof, and in which itis shown by way of illustration specific embodiments in which theinvention may be practiced. It is to be understood that otherembodiments may be utilized and structural changes may be made withoutdeparting from the scope of the preferred embodiments of the presentinvention.

This relates to a touch event model that simplifies the recognition ofsingle and multiple touch events for user interface applications runningin multi-point and multi-touch enabled devices. To simplify therecognition of single and multiple touch events, each view within aparticular window can be configured as either a multi-touch view or asingle touch view. Furthermore, each view can be configured as either anexclusive or a non-exclusive view. Depending on the configuration of aview, touch events in that and other views can be either ignored orrecognized.

Although embodiments of the present invention may be described andillustrated herein in terms of specific multi-touch capable devices, itshould be understood that embodiments of the present invention are notlimited to such devices, but is generally applicable to any multi-touchcapable device. Furthermore, embodiments of the invention are notlimited to multi-touch devices but also include multi-point devices,such as multi proximity sensor devices as discussed in U.S. applicationSer. No. 11/649,998, mentioned above.

Some embodiments are related to APIs. In general, an API is a sourcecode interface that a computer system provides in order to supportrequests for services from a software operation. An API is specified interms of a program language that can be interpreted or compiled when asystem is built, rather than an explicit low level description of howdata is laid out in memory. The software that provides the functionalityof an API is said to be an implementation of the API. Various devicessuch as computer systems, electronic devices, portable devices andhandheld devices have software applications. The device interfacesbetween the software applications and user interface software to providea user of the device with certain features and operations.

At least some embodiments of the invention can include one or more APIsin an environment with user interface software interacting with asoftware application. Various function calls or messages are transferredvia the APIs between the user interface software and the softwareapplications. Transferring the function calls or messages may includeissuing, initiating, invoking or receiving the function calls ormessages. Example APIs can include sending touch event information. AnAPI may also implement functions having parameters, variables orpointers. An API may receive parameters as disclosed or othercombinations of parameters. In addition to the APIs disclosed, otherAPIs individually or in combination can perform similar functionality asthe disclosed APIs.

FIG. 1 is a diagram of an input/output processing stack of an exemplarymulti-touch capable device according to some embodiments of theinvention. Hardware 100 can be provided at the base level of amulti-touch enabled device. It can include various hardware interfacecomponents, such as a multi-touch enabled panel 101 and/or anaccelerometer 102. The multi-touch panel can include a display and apanel that senses multiple touches simultaneously. An example of such apanel is discussed in more detail in the Ser. No. 11/649,998 applicationmentioned above. The accelerometer can be a hardware device that sensesacceleration of the multi-touch enabled device. It can be used to sensewhen the device is being moved, how it is being moved, if it is dropped,etc. Other hardware interface devices, such as gyroscopes, speakers,buttons, infrared (IR) sensors, etc. (not shown) can also be included.

A driver or a set of drivers 103 can communicate with the hardware 100.The drivers can receive and process input data from received from thehardware. A core Operating System (OS) 104 can communicate with thedriver(s). The core OS can process raw input data received from thedriver(s). In some embodiments, the drivers can be considered to be apart of the core OS.

A set of OS application programming interfaces (APIs) 105 cancommunicate with the core OS. These APIs can be a set of APIs that areusually included with operating systems (such as, for example, Linux orUNIX APIs). User Interface APIs 106 (UI APIs) can include a set of APIsdesigned for use by applications running on the device. The UI APIs canutilize the OS APIs. Applications 107 running on the device can utilizethe APIs of the UI APIs in order to communicate with the user. The UIAPIs can, in turn, communicate with lower level elements, ultimatelycommunicating with the multi-touch panel 101 and various other userinterface hardware. While each layer can utilize the layer underneathit, that is not always required. For example, in some embodiments,applications 107 can occasionally communicate with OS APIs 105. APIs 105and 106 can comprise respective sets of application programminginterfaces as well as their respective implementations. For example, UIAPIs 106 can also include user interface (UI) software for implementingthe UI APIs.

FIGS. 2A and 2B are diagrams of two types of exemplary multi-touchenabled devices according to some embodiments of the invention. FIG. 2Ashows exemplary device 200. Device 200 can include a CPU 201 and amemory 202 connected through a bus 204. The bus can also connect to amulti-touch display 203. The multi-touch display can include amulti-touch panel and a display. The multi-touch panel and the displaycan be combined to form the multi-touch display 203. The multi-touchdisplay can correspond to the multi-touch panel 101 within hardwarelayer 100 of FIG. 1. The CPU can be used to execute software stored inthe memory. The software executed by the CPU can include layers 103-109of FIG. 1. Thus, the software can include drivers, an OS, various APIsand applications.

FIG. 2B shows alternative device 210. Device 210 can be similar todevice 200. However, device 210 can include a separate multi-touch panel(212) and display (211) instead of the single unit of device 200. Thus,for device 210 one need not touch the display in order to interact withthe multi-touch panel. Device 210 can be, for example, a multi-touchtrack-pad equipped laptop computer (the multi-touch panel serving as atrack pad).

The multi touch panel and/or display of FIGS. 2A and 2B can also utilizeother sensory technology, such as proximity sensing, as discussed inU.S. application Ser. No. 11/649,998, mentioned above. Generally, amulti-point panel and/or display can be used for the devices of FIGS. 2Aand 2B. The multi-point panel and/or display can feature various typesof sensor technology. For example, it can feature multi-touch technologyonly (thus resulting in a multi-touch panel and/or display),multi-proximity sense technology, a combination of the two, or anothertype of multi-point technology.

The devices of FIGS. 2A and 2B can include various different types ofmulti-touch enabled devices. For example, they can include a mobiletelephone, a portable video game console, an electronic music player, ane-book, a PDA, an electronic organizer, an e-mail device, a laptop orother personal computer, a kiosk computer, a vending machine, etc.

FIG. 3 is a diagram of an exemplary multi-touch display 300. Themulti-touch display can be display 203 of FIG. 2A or display 211 of FIG.2B. The display can display various user interface elements (such asgraphics, etc.) generated by software running in the deviceincorporating the display (e.g., device 200 of FIG. 2A or device 210 ofFIG. 2B). The user can interact with the various user interface elementsin order to interact with the software. When using the device of FIG.2A, the user can interact with the user interface elements by touchingthem directly on the display. When using the device of FIG. 2B, the usercan touch the separate multi-touch panel 212 in order to move andcontrol one or more cursors on the display 211, the cursors being usedto interact with the software.

The user interface elements rendered at the display 300 can include oneor more views. Each view can represent a graphical user interfaceelement handled by a separate software element. The separate softwareelements can include different applications, different processes orthreads (even if within the same application), different routines orsubroutines, different objects, etc. In some embodiments, each separatesoftware element can create user interface elements for its respectiveportion of the display as well as receive and handle touch inputs forthat portion of the display. The touch inputs can be processed by thevarious layers discussed in connection with FIG. 1, which cansubsequently send processed touch input data to the software element(which can be part of applications 109). The processed touch input datacan be referred to as touch event(s) and can be in a format that iseasier to handle than raw touch data generated by the multi-touch panel.For example, each touch event can include a set of coordinates at whicha touch is currently occurring. In some embodiments, the set ofcoordinates may correspond to the centroid of a touch. For the sake ofbrevity and simplicity, the discussion below may refer to a softwareelement associated with a view by simply referring to the view itself.

Views can be nested. In other words, a view can include other views.Consequently, the software element associated with a first view caninclude or be linked to one or more software elements associated withviews within the first view. While some views can be associated withapplications, others can be associated with high level OS elements, suchas graphical user interfaces, window managers, etc.

The exemplary display of FIG. 3 shows a music browsing application. Thedisplay can include a status bar view 301 that indicates the overallstatus of the device. The status bar view can be part of the OS. Titleview 302 can also be included. The title view can itself include severalother views, such as center title view 310, back button 312 and forwardbutton 311. Table view 303 can also be included. Table view 303 caninclude one or more table element views, such as table element view 304.As seen, in one embodiment, the table element views can be song titles.A button bar view 305 can also be included. The button bar view caninclude buttons 306-309.

Each view and its associated software element may be able to receive,process and handle touch events that occur at that particular view.Thus, for example, if a user touches song title view 304, the softwareelement associated with that view can receive a touch event indicatingthat the view has been touched, process it and respond accordingly. Forexample, the software element can change the graphical representation ofthe view (i.e., highlighting the view), and/or cause other actions suchas playing a song associated with the touched view.

In some embodiments, touch events are processed at the lowest level ofthe view hierarchy. Thus, for example, if a user touches title bar view302, the touch event need not be directly processed by the softwareelement associated with the title bar view, but instead can be processedby a software element associated with a view included within the titlebar view where the touch occurred (i.e., a software element associatedwith one of views 310, 311 and 312). In some embodiments, some higherlevel views can also handle touch events. In addition, various softwareelements that are not associated with a view being touched cannevertheless be alerted or can discover that the view is being touched.

Since display 300 is a multi-touch display, multiple touches can occurat the same time. The multiple touches can occur in the same view, or intwo or more different views. Furthermore, the user can perform gestures(e.g., by pressing down one or more fingers and moving them) that canhave predefined meanings. Multi-touch gestures are discussed in moredetail in U.S. patent application Ser. No. 10/903,964, entitled“Gestures for Touch Sensitive Input Devices,” filed on Jul. 30, 2004 andhereby incorporated by reference in its entirety.

A view can receive touch events that start within the view. If a userkeeps a finger pressed against the display, then the view can receivemultiple touch events indicating a continuous touch. If a user moves apressed finger, the view can receive multiple touch events indicatingmovement of the touch. If a user moves a pressed finger outside of theview, then the view can still receive touch events associated with thatmovement (and the views to which the finger has been moved need notreceive such touch events). Thus, a view can receive events associatedwith a gesture or a movement that begins at the view, even if itcontinues outside of the view.

A touch can refer to an act which begins with pressing a finger oranother body part or object to the surface of a multi touch panel (ormulti touch display) and ends when the finger or object is removed fromthe display. Thus, the touch can include moving of the finger or object,or keeping the finger or object at the same place for a period of time.

Touch events can be sent to views (or the software elements thatimplement the views) by one or more APIs (and their respectiveimplementations). An example of an API for handling touch events isprovided in Appendix A below. According to the API of Appendix A, theAPI can send each view a touch event data structure that includes one ormore single touch data structures (or touch data structures). Each touchevent data structure can define the current state of all touches takingplace at the view at a particular moment in time. The respective touchdata structures within a touch event data structure can define thecurrent states of one or more respective single touches at theparticular moment in time. Thus, if there are three touches taking placeat a particular moment in time in a particular view, a touch event datastructure comprising three touch data structures defining the states ofthe three touches can be sent to the view. In some embodiments, touchdata structures can be sent even if their associated touches are nolonger taking place in order to alert the view that the touches haveterminated.

As noted above, a touch may include an act that need not beinstantaneous. E.g., a touch can include an act of moving or holding afinger against a display for a period of time. A touch data structure,however, defines a state of a touch at a particular time. Therefore,multiple touch data structures may be associated with a single touch,thus defining the single touch at different points in time.

Each touch data structure can comprise various fields. A “first touchfor view” field can indicate whether the touch data structure definesthe first touch for the particular view (since the software elementimplementing the view was instantiated). A “time stamp” field canindicate the particular time that the touch data structure relates to.

An “info” field can be used to indicate if a touch is a rudimentarygesture. For example, the “info” field can indicate whether the touch isa swipe and, if so, in which direction the swipe is oriented. A swipe isa quick drag of one or more fingers in a straight direction. The APIimplementations can determine if a touch is a swipe and pass thatinformation to the application through the “info” field, thusalleviating the application of some data processing that would have beenotherwise necessary.

A “tap count” field can indicate how many taps have been sequentiallyperformed at the position of the touch. A tap can be defined as a quickpressing and lifting of a finger against a panel at a particularposition. Multiple sequential taps can occur if the finger is againpressed and released in quick succession at the same position of thepanel. Thus, the API implementation can count taps for variousapplication and relay this information through the tap “count field.”Multiple taps at the same location are sometimes considered to be a veryuseful and easy to remember command for touch enabled interfaces. Thus,by counting taps, the API can again alleviate some data processing fromthe application.

A “phase” field can indicate a particular phase the touch is currentlyin. The phase field can have various values, such as “touch phase began”which can indicate that the touch data structure defines a new touchthat has not been referenced by previous touch data structures. A “touchphase moved” value can indicate that the touch being defined has movedfrom a position defined in a previous touch data structure. A “touchphase stationary” value can indicate that the touch has stayed in thesame position since the last touch data structure for that touch wasgenerated. A “touch phase ended” value can indicate that the touch hasended (e.g., the user has lifted his/her finger from the surface of amulti touch display). A “touch phase cancelled” value can indicate thatthe touch has been cancelled by the device. A cancelled touch can be atouch that is not necessarily ended by a user, but which the device candetermine to ignore. For example, the device can determine that thetouch is being generated inadvertently (i.e., as a result of placing aportable multi touch enabled device in one's pocket) and ignore thetouch for that reason. Each value of the “phase field” can be an integernumber.

Thus, each touch data structure can define what is happening with atouch at a particular time (e.g., whether the touch is stationary, beingmoved, etc.) as well as other information associated with the touch(such as position). Accordingly, each touch data structure can definethe state of a particular touch at a particular moment in time. One ormore touch data structures referencing the same time can be added in atouch event data structure that can define the states of all touches aparticular view is receiving at a moment in time (as noted above, sometouch data structures may also reference touches that have ended and areno longer being received). Multiple touch event data structures can besent to the software implementing a view as time passes, in order toprovide the software with continuous information describing the touchesthat are happening at the view. One or more elements of the device suchas, for example, hardware 100, drivers 103, core OS 104, OS APIs 105 andUI APIs can detect touches at the multi touch panel 101 and generate thevarious touch event data structures defining these touches.

The ability to handle multiple touches and multi-touch gestures can addcomplexity to the various software elements. In some cases, suchadditional complexity can be necessary to implement advanced anddesirable interface features. For example, a game may require theability to handle multiple simultaneous touches that occur in differentviews, as games often require the pressing of multiple buttons at thesame time. However, some simpler applications and/or views (and theirassociated software elements) need not require advanced interfacefeatures. For example, a simple button (such as button 306) can besatisfactorily operable with single touches and need not requiremulti-touch functionality. In these cases, the underlying OS may sendunnecessary or excessive touch data (e.g., multi-touch data) to asoftware element associated with a view that is intended to be operableby single touches only (e.g., a button). Because the software elementmay need to process this data, it may need to feature all the complexityof a software element that handles multiple touches, even though it isassociated with a view for which only single touches are relevant. Thiscan increase the cost of development of software for the device, becausesoftware elements that have been traditionally very easy to program in amouse interface environment (i.e., various buttons, etc.) may be muchmore complex in a multi-touch environment.

Embodiments of the present invention address the above discussed issuesby selectively providing touch data to various software elements inaccordance with predefined settings. Thus, a simpler interface can beprovided for selected software elements, while others can take advantageof more complex multi-touch input.

Embodiments of the invention can rely on one or more flags associatedwith one or more views, wherein each flag or combination thereofindicates a mode of touch event processing for a particular view. Forexample, multi-touch and/or exclusive touch flags can be used. Themulti-touch flag can indicate whether a particular view is capable ofreceiving multiple simultaneous touches or not. The exclusive touch flagcan indicate whether a particular view is to allow other views toreceive touch events while the view is receiving a touch event.

FIG. 4 is a flow chart showing the operation of the multi-touch flagaccording to one embodiment of the invention. At step 400, a user cantouch a view at a first location within the view. It can be assumed thatno other touches are present on the multi-touch display when the touchof step 400 is received. At step 402, the OS can send a touch eventdefining the received touch to a software element associated with thetouched location.

At step 404, the user can touch the view at a second location while notreleasing the first touch (i.e., while keeping a finger pressed down atthe first location). Thus, for example, the user can touch the rightportion of table element view 304 at step 400 and touch the left portionof table element view 304 at step 404 without releasing his/her fingerfrom the right portion. Therefore, the second touch is contemporaneouswith the first touch (thus taking advantage of the multi-touchcapabilities of display 300).

At step 406, the OS can determine whether the multi-touch flag for theview being touched is set. If the multi-touch flag is set, then the viewcan be a view that can handle multiple contemporaneous touches.Therefore, at step 408, a second touch event for the second touch can besent to the software element associated with the view. It should benoted that new instances of the first touch event can also be sent,indicating that the first touch event is still taking place (i.e., thefinger at the first location has not been lifted). The new instances ofthe first touch event can specify different locations if the finger atthe first location is moved away from that location without being lifted(i.e., if it is being “dragged” on the surface of the display).

If, on the other hand, the multi-touch flag is not set, the OS canignore or block the second touch. Ignoring the second touch can resultin not sending any touch events associated with the second touch to thesoftware element associated with the touched view. In some embodiments,the OS can alert other software elements of the second touch, ifnecessary.

Thus, embodiments of the present invention can allow relatively simplesoftware elements that are programmed to handle only a single touch at atime to keep their multi-touch flag unasserted, and thus ensure thattouch events that are part of multiple contemporaneous touches will notbe sent to them. Meanwhile, more complex software elements that canhandle multiple contemporaneous touches can assert their multi-touchflag and receive touch events for all touches that occur at theirassociated views. Consequently, development costs for the simplesoftware elements can be reduced while providing advanced multi-touchfunctionality for more complex elements.

FIGS. 5A and 5B are a flow chart showing an exemplary method ofoperation of the exclusive touch flag according to one embodiment of theinvention. At step 500, a user can touch a first view. At step 502, theOS can send a touch event to a first software element associated withthe first view. At step 504, the user can touch a second view withoutreleasing the first touch.

At step 506, the OS can check whether the exclusive touch flag for thefirst view is asserted. If it is set (asserted), that means that thefirst view needs to receive touches exclusively, and no other touchesare to be sent to other views. Thus, if the exclusive touch flag is set,the OS can ignore (or block) the second touch and not send it to anysoftware elements. If the exclusive view flag is not set, then theprocess can continue to step 510 of FIG. 5B.

In step 510, the OS can determine if the exclusive view flag for thesecond view is set. If that flag is set, then the second view can onlyreceive exclusive touch events. Thus, if there is another touch eventalready being received by another view (i.e., the first view), thesecond view cannot receive a touch event, and the OS can ignore thesecond touch (step 512). However, if the exclusive touch flag for thesecond touch is not set (unasserted), the OS can send a touch eventassociated with the second touch to the second view. More specifically,the OS can send a touch event associated with the second touch to asoftware element associated with the second view (step 514).

Thus, the exclusive touch flag can ensure that views flagged asexclusive only receive touch events when they are the only views on thedisplay receiving touch events. The exclusive flag can be very useful insimplifying the software of applications running on a multi-touchenabled device. In certain situations, allowing multiple views toreceive touches simultaneously can result in complex conflicts anderrors. For example, if a button to delete a song and a button to play asong are simultaneously pressed, this may cause an error. Avoiding suchconflicts may require complex and costly software. However, embodimentsof the present invention can reduce the need for such software byproviding an exclusive touch flag which can ensure that a view that hasthat flag set will receive touch events only when it is the only viewthat is receiving a touch event. Alternatively, one or more views canhave their exclusive touch flags unasserted, thus allowing multiplesimultaneous touches at two or more of these views.

In some embodiments the exclusive flag can signify exclusivity for theentire display. Thus, when a view with the exclusive flag set isreceiving a touch event, all other views in the display can be blockedfrom receiving any touch events. In alternative embodiments, theexclusive flag can signify exclusivity in a smaller area such as asingle application, or a single window. For example, a first view withits exclusivity flag set can block other views that are in the samewindow from receiving any touch events while the first view is receivinga touch event, but not block views in other windows.

The exclusive touch and multi-touch flags can be combined. Accordingly,one or more views being displayed can each include two flags—amulti-touch flag and an exclusive touch flag. In some embodiments, alldisplayed views can include these two flags. The value of one flag neednot depend on the value of another. In one example, a view with bothexclusive and multi-touch flags set can allow multiple touches withinthe view but may only receive touches exclusively (i.e., when the viewis receiving touches, touches to other views can be blocked). A viewwith both flags unasserted can block multiple touches within the viewbut allow single touches within the view even if touches aresimultaneously taking place in other views. A view with the multi-touchflag unasserted and the exclusive touch flag asserted can allow onlysingle touches within the view when no other touches are taking place inany other views. A view with the multi-touch flag asserted and theexclusive touch flag unasserted can allow all touches received for theview. A view with both flags asserted can allow multiple touches in theview while no other touches are taking place for other views.

Alternative embodiments can feature only one of the flags (and theassociated functionality). Thus, some embodiments can use themulti-touch flag only or exclusive touch flag only. In some embodiments,different views can use different combinations of the flags.

The various functionalities performed by the OS in FIGS. 4, 5A and 5Bcan instead be performed by other software, such as various utilitysoftware. These functionalities can be performed by software at any onelayer of layers 103 through 108 of FIG. 1. In an alternative embodiment,these functionalities can even be performed by hardware 100.

Provided below is an exemplary set of code showing the methods of anexemplary software element associated with a view according to someembodiments of the invention. A person of skill in the art wouldrecognize that other code may also be used to implement thefunctionalities discussed above.

While the above discussion centers on multi-touch displays and panels,the present invention is not limited to multi-touch device but mayinclude various multi-point devices as discussed above (including, forexample, multi-proximity sensor devices). For multi-point devices,multi-point and an exclusive point flags can be used. These flags canoperate in a similar manner to the multi-touch and exclusive touch flagsdiscussed above.

Although the present invention has been fully described in connectionwith embodiments thereof with reference to the accompanying drawings, itis to be noted that various changes and modifications will becomeapparent to those skilled in the art. Such changes and modifications areto be understood as being included within the scope of the presentinvention as defined by the appended claims.

What is claimed is:
 1. A non-transitory computer readable medium storingsoftware that when executed by an electronic device, causes theelectronic device to perform operations comprising: executing a softwareelement representing a view, the software element displaying a visualrepresentation of the view at a display; detecting a touch at amulti-touch panel; and in response to detecting the touch, generating atouch data structure for defining a state of the touch at a particulartime, the touch data structure comprising: a location field indicating alocation where the touch was or is being received on the multi-touchpanel; a phase field defining a phase of the touch at the particulartime, the phase of the touch being selected from a plurality ofpredefined phase state values; and a view field indicating a respectiveview, of a plurality of views corresponding to portions of a displayeduser interface, that corresponds to the location where the touch was oris being received on the multi-touch panel; and providing the touch datastructure to the software element representing the view.
 2. Thenon-transitory computer readable medium of claim 1, including softwarethat when executed by an electronic device, causes the electronic deviceto provide, to the software element representing the view, a touch eventdata structure indicating the state, at a particular time, of two ormore touches being or having been simultaneously received at themulti-touch panel, the touch event data structure comprising a pluralityof instances of the touch data structure, each touch data structurebeing associated with one of the two or more touches.
 3. Thenon-transitory computer readable medium of claim 1, wherein the touchdata structure further comprises: a time field indicating the particulartime; a window field indicating a window in which the touch was or isbeing received; a previous location field indicating a location at whichthe touch was previously received; and a first touch for view fieldindicating whether the touch is the first one received for the view inwhich it was or is being received.
 4. The non-transitory computerreadable medium of claim 1, wherein the touch data structure furthercomprises an info field indicating whether the detected touch comprisesa swiping motion and, if it does, a general direction along which themotion is aligned.
 5. The non-transitory computer readable medium ofclaim 1, wherein the phase field holds a value from among a set ofpredefined touch phase values, the set of predefined touch phase valuesincluding a touch began value, indicating that the touch is a new touchthat had been initiated at the particular time, and a touch moved value,indicating that the touch has been moved from a previous position to anew position.
 6. The non-transitory computer readable medium of claim 5,wherein the set of predefined touch phase values includes a touchstationary value indicating that the touch has stayed stationary sincethe generation of a previous touch data structure for the touch.
 7. Thenon-transitory computer readable medium of claim 5, wherein the set ofpredefined touch phase values includes a touch ended value indicatingthat the touch has ended.
 8. The non-transitory computer readable mediumof claim 5, wherein the set of predefined touch phase values includes atouch canceled value indicating that the touch has been cancelled by theelectronic device, of which the multi-touch panel forms part.
 9. Thenon-transitory computer readable medium of claim 1, wherein the touchdata structure further comprises a tap count field indicating a numberof short taps that have been sequentially performed at the location ofthe touch.
 10. An electronic device comprising a multi-touch panel and anon-transitory computer readable medium storing software that whenexecuted by the electronic device, causes the electronic device toperform operations comprising: executing a software element representinga view, the software element displaying a visual representation of theview at a display; detecting a touch at a multi-touch panel; and inresponse to detecting the touch, generating a touch data structure fordefining a state of the touch at a particular time, the touch datastructure comprising: a location field indicating a location where thetouch was or is being received on the multi-touch panel; a phase fielddefining a phase of the touch at the particular time, the phase of thetouch being selected from a plurality of predefined phase state values;and a view field indicating a respective view, of a plurality of viewscorresponding to portions of a displayed user interface, thatcorresponds to the location where the touch was or is being received onthe multi-touch panel; and providing the touch data structure to thesoftware element representing the view.
 11. The electronic device ofclaim 10, wherein the non-transitory computer readable medium includessoftware that when executed by the electronic device, causes theelectronic device to provide, to the software element representing theview, a touch event data structure indicating the state, at a particulartime, of two or more touches being or having been simultaneouslyreceived at the multi-touch panel, the touch event data structurecomprising a plurality of instances of the touch data structure, eachtouch data structure being associated with one of the two or moretouches.
 12. The electronic device of claim 10, wherein the touch datastructure further comprises: a time field indicating the particulartime; a window field indicating a window in which the touch was or isbeing received; a previous location field indicating a location at whichthe touch was previously received; and a first touch for view fieldindicating whether the touch is the first one received for the view inwhich it was or is being received.
 13. The electronic device of claim10, wherein the touch data structure further comprises an info fieldindicating whether the detected touch comprises a swiping motion and, ifit does, a general direction along which the motion is aligned.
 14. Theelectronic device of claim 10, wherein the phase field holds a valuefrom among a set of predefined touch phase values, the set of predefinedtouch phase values including a touch began value, indicating that thetouch is a new touch that had been initiated at the particular time, anda touch moved value, indicating that the touch has been moved from aprevious position to a new position.
 15. The electronic device of claim14, wherein the touch data structure further comprises a tap count fieldindicating a number of short taps that have been sequentially performedat the location of the touch.
 16. The electronic device of claim 14,wherein the set of predefined touch phase values includes a touchstationary value indicating that the touch has stayed stationary sincethe generation of a previous touch data structure for the touch.
 17. Theelectronic device of claim 14, wherein the set of predefined touch phasevalues includes a touch ended value indicating that the touch has ended.18. The electronic device of claim 14, wherein the set of predefinedtouch phase values includes a touch canceled value indicating that thetouch has been cancelled by the electronic device, of which themulti-touch panel forms part.
 19. A method for operating an electronicdevice, comprising: executing, by a processor of the electronic device,a software element representing a view, the software element displayinga visual representation of the view at a display; detecting a touch at amulti-touch panel; and in response to detecting the touch, generating,by the processor of the electronic device, a touch data structure fordefining a state of the touch at a particular time, the touch datastructure comprising: a location field indicating a location where thetouch was or is being received on the multi-touch panel; a phase fielddefining a phase of the touch at the particular time, the phase of thetouch being selected from a plurality of predefined phase state values;and a view field indicating a respective view, of a plurality of viewscorresponding to portions of a displayed user interface, thatcorresponds to the location where the touch was or is being received onthe multi-touch panel; and providing, by the processor of the electronicdevice, the touch data structure to the software element representingthe view.
 20. The method of claim 19, including providing, by theprocessor of the electronic device, to the software element representingthe view, a touch event data structure indicating the state, at aparticular time, of two or more touches being or having beensimultaneously received at the multi-touch panel, the touch event datastructure comprising a plurality of instances of the touch datastructure, each touch data structure being associated with one of thetwo or more touches.
 21. The method of claim 19, wherein the touch datastructure further comprises: a time field indicating the particulartime; a window field indicating a window in which the touch was or isbeing received; a previous location field indicating a location at whichthe touch was previously received; and a first touch for view fieldindicating whether the touch is the first one received for the view inwhich it was or is being received.
 22. The method of claim 19, whereinthe touch data structure further comprises an info field indicatingwhether the detected touch comprises a swiping motion and, if it does, ageneral direction along which the motion is aligned.
 23. The method ofclaim 19, wherein the phase field holds a value from among a set ofpredefined touch phase values, the set of predefined touch phase valuesincluding a touch began value, indicating that the touch is a new touchthat had been initiated at the particular time, and a touch moved value,indicating that the touch has been moved from a previous position to anew position.
 24. The method of claim 23, wherein the set of predefinedtouch phase values includes a touch stationary value indicating that thetouch has stayed stationary since the generation of a previous touchdata structure for the touch.
 25. The method of claim 23, wherein theset of predefined touch phase values includes a touch ended valueindicating that the touch has ended.
 26. The method of claim 23, whereinthe set of predefined touch phase values includes a touch canceled valueindicating that the touch has been cancelled by the electronic device,of which the multi-touch panel forms part.
 27. The method of claim 19,wherein the touch data structure further comprises a tap count fieldindicating a number of short taps that have been sequentially performedat the location of the touch.